Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.26%. Comparing base (2691e7c) to head (0049bee).

@@           Coverage Diff           @@
##             main    #1383   +/-   ##
=======================================
  Coverage   82.26%   82.26%           
=======================================
  Files          50       50           
  Lines        3535     3535           
=======================================
  Hits         2908     2908           
  Misses        627      627           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Hlw @yebai, this PR partially addresses #1374.

I've gone through the issue carefully and made the following changes

• Removed the arbitrary reference time baseline (simple_assume_observe_non_model)
• t(eval) is now reported in absolute nanoseconds, which is more meaningful and interpretable
• t(grad)/t(eval) ratio is kept as-is since it was already useful
• Updated column headers accordingly

I also noted @penelopeysm's point that introducing an external baseline doesn't simplify comparative benchmarks you still need two checkouts regardless. So I've kept this PR focused on the absolute time reporting for now.

Note: The Benchmarking / combine results check is failing but this appears to be a pre existing CI issue unrelated to this PR.

Introducing Stan as an external baseline would be a separate more involved task happy to work on that as a follow up PR.

Closing my PR #1383 as the upstream has since merged a more comprehensive benchmark improvement in #1385. The absolute time reporting approach from my PR has been superseded.